Distributed data structure templates for data‐intensive remote sensing applications
Identifieur interne : 000421 ( Main/Exploration ); précédent : 000420; suivant : 000422Distributed data structure templates for data‐intensive remote sensing applications
Auteurs : Yan Ma [République populaire de Chine] ; Lizhe Wang [République populaire de Chine] ; Dingsheng Liu [République populaire de Chine] ; Tao Yuan [République populaire de Chine] ; Peng Liu [République populaire de Chine] ; Wanfeng Zhang [République populaire de Chine]Source :
- Concurrency and Computation: Practice and Experience [ 1532-0626 ] ; 2013-08-25.
Abstract
The remotely sensed images continuously acquired by satellite and airborne sensors are increasing dramatically. Remote sensing applications are overwhelmed with tons of remote sensing data with complex data structures. Efficient programming in parallel systems for data‐intensive applications like massive remote sensing data processing will be a challenge. We propose a generic data‐structure oriented programming template to support massive remote sensing data processing in high‐performance clusters. These templates provide distributed abstractions for large remote sensing image data with complex data structure and allow these distributed data to be accessed as a global one. Through data serialization and one‐sided message passing primitives provided by message passing interface, the distributed remote sensing data template whose sliced data blocks are scattered among nodes could offer a simple and effective way to distribute and communicate massive remote sensing data. Efficient parallel input/output directly to and from the distributed data structure will also be offered to address the input/output bottleneck caused by massive image data. Developers can take the advantage of our templates to program efficient parallel remote sensing algorithms without dealing with data slicing and communication through low‐level message passing interface APIs. Through experiments on remote sensing applications, we confirmed that our templates were productive and efficient. Copyright © 2012 John Wiley & Sons, Ltd.
Url:
DOI: 10.1002/cpe.2965
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream Istex, to step Corpus: 000418
- to stream Istex, to step Curation: 000418
- to stream Istex, to step Checkpoint: 000040
- to stream Main, to step Merge: 000422
- to stream Main, to step Curation: 000421
Le document en format XML
<record><TEI wicri:istexFullTextTei="biblStruct"><teiHeader><fileDesc><titleStmt><title xml:lang="en">Distributed data structure templates for data‐intensive remote sensing applications</title><author><name sortKey="Ma, Yan" sort="Ma, Yan" uniqKey="Ma Y" first="Yan" last="Ma">Yan Ma</name></author><author><name sortKey="Wang, Lizhe" sort="Wang, Lizhe" uniqKey="Wang L" first="Lizhe" last="Wang">Lizhe Wang</name></author><author><name sortKey="Liu, Dingsheng" sort="Liu, Dingsheng" uniqKey="Liu D" first="Dingsheng" last="Liu">Dingsheng Liu</name></author><author><name sortKey="Yuan, Tao" sort="Yuan, Tao" uniqKey="Yuan T" first="Tao" last="Yuan">Tao Yuan</name></author><author><name sortKey="Liu, Peng" sort="Liu, Peng" uniqKey="Liu P" first="Peng" last="Liu">Peng Liu</name></author><author><name sortKey="Zhang, Wanfeng" sort="Zhang, Wanfeng" uniqKey="Zhang W" first="Wanfeng" last="Zhang">Wanfeng Zhang</name></author></titleStmt><publicationStmt><idno type="wicri:source">ISTEX</idno><idno type="RBID">ISTEX:6B02BA7B32D997BB4BE8B5F238D7607ECDB8CDAB</idno><date when="2013" year="2013">2013</date><idno type="doi">10.1002/cpe.2965</idno><idno type="url">https://api.istex.fr/document/6B02BA7B32D997BB4BE8B5F238D7607ECDB8CDAB/fulltext/pdf</idno><idno type="wicri:Area/Istex/Corpus">000418</idno><idno type="wicri:Area/Istex/Curation">000418</idno><idno type="wicri:Area/Istex/Checkpoint">000040</idno><idno type="wicri:doubleKey">1532-0626:2013:Ma Y:distributed:data:structure</idno><idno type="wicri:Area/Main/Merge">000422</idno><idno type="wicri:Area/Main/Curation">000421</idno><idno type="wicri:Area/Main/Exploration">000421</idno></publicationStmt><sourceDesc><biblStruct><analytic><title level="a" type="main" xml:lang="en">Distributed data structure templates for data‐intensive remote sensing applications</title><author><name sortKey="Ma, Yan" sort="Ma, Yan" uniqKey="Ma Y" first="Yan" last="Ma">Yan Ma</name><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Center for Earth Observation and Digital Earth, Chinese Academy of Sciences</wicri:regionArea><wicri:noRegion>Chinese Academy of Sciences</wicri:noRegion></affiliation><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Institute of Electronics, Chinese Academy of Sciences</wicri:regionArea><wicri:noRegion>Chinese Academy of Sciences</wicri:noRegion></affiliation><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Graduate University of Chinese Academy of Sciences</wicri:regionArea></affiliation></author><author><name sortKey="Wang, Lizhe" sort="Wang, Lizhe" uniqKey="Wang L" first="Lizhe" last="Wang">Lizhe Wang</name><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Center for Earth Observation and Digital Earth, Chinese Academy of Sciences</wicri:regionArea><wicri:noRegion>Chinese Academy of Sciences</wicri:noRegion></affiliation><affiliation><wicri:noCountry code="subField">China.E‐mail:</wicri:noCountry></affiliation><affiliation><wicri:noCountry code="no comma">E-mail: lizhe.wang@gmail.com</wicri:noCountry></affiliation></author><author><name sortKey="Liu, Dingsheng" sort="Liu, Dingsheng" uniqKey="Liu D" first="Dingsheng" last="Liu">Dingsheng Liu</name><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Center for Earth Observation and Digital Earth, Chinese Academy of Sciences</wicri:regionArea><wicri:noRegion>Chinese Academy of Sciences</wicri:noRegion></affiliation></author><author><name sortKey="Yuan, Tao" sort="Yuan, Tao" uniqKey="Yuan T" first="Tao" last="Yuan">Tao Yuan</name><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Center for Earth Observation and Digital Earth, Chinese Academy of Sciences</wicri:regionArea><wicri:noRegion>Chinese Academy of Sciences</wicri:noRegion></affiliation><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Graduate University of Chinese Academy of Sciences</wicri:regionArea></affiliation></author><author><name sortKey="Liu, Peng" sort="Liu, Peng" uniqKey="Liu P" first="Peng" last="Liu">Peng Liu</name><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Center for Earth Observation and Digital Earth, Chinese Academy of Sciences</wicri:regionArea><wicri:noRegion>Chinese Academy of Sciences</wicri:noRegion></affiliation></author><author><name sortKey="Zhang, Wanfeng" sort="Zhang, Wanfeng" uniqKey="Zhang W" first="Wanfeng" last="Zhang">Wanfeng Zhang</name><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Center for Earth Observation and Digital Earth, Chinese Academy of Sciences</wicri:regionArea><wicri:noRegion>Chinese Academy of Sciences</wicri:noRegion></affiliation><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Graduate University of Chinese Academy of Sciences</wicri:regionArea></affiliation></author></analytic><monogr></monogr><series><title level="j">Concurrency and Computation: Practice and Experience</title><title level="j" type="abbrev">Concurrency Computat.: Pract. Exper.</title><idno type="ISSN">1532-0626</idno><idno type="eISSN">1532-0634</idno><imprint><publisher>Blackwell Publishing Ltd</publisher><date type="published" when="2013-08-25">2013-08-25</date><biblScope unit="volume">25</biblScope><biblScope unit="issue">12</biblScope><biblScope unit="page" from="1784">1784</biblScope><biblScope unit="page" to="1797">1797</biblScope></imprint><idno type="ISSN">1532-0626</idno></series><idno type="istex">6B02BA7B32D997BB4BE8B5F238D7607ECDB8CDAB</idno><idno type="DOI">10.1002/cpe.2965</idno><idno type="ArticleID">CPE2965</idno></biblStruct></sourceDesc><seriesStmt><idno type="ISSN">1532-0626</idno></seriesStmt></fileDesc><profileDesc><textClass></textClass><langUsage><language ident="en">en</language></langUsage></profileDesc></teiHeader><front><div type="abstract">The remotely sensed images continuously acquired by satellite and airborne sensors are increasing dramatically. Remote sensing applications are overwhelmed with tons of remote sensing data with complex data structures. Efficient programming in parallel systems for data‐intensive applications like massive remote sensing data processing will be a challenge. We propose a generic data‐structure oriented programming template to support massive remote sensing data processing in high‐performance clusters. These templates provide distributed abstractions for large remote sensing image data with complex data structure and allow these distributed data to be accessed as a global one. Through data serialization and one‐sided message passing primitives provided by message passing interface, the distributed remote sensing data template whose sliced data blocks are scattered among nodes could offer a simple and effective way to distribute and communicate massive remote sensing data. Efficient parallel input/output directly to and from the distributed data structure will also be offered to address the input/output bottleneck caused by massive image data. Developers can take the advantage of our templates to program efficient parallel remote sensing algorithms without dealing with data slicing and communication through low‐level message passing interface APIs. Through experiments on remote sensing applications, we confirmed that our templates were productive and efficient. Copyright © 2012 John Wiley & Sons, Ltd.</div></front></TEI><affiliations><list><country><li>République populaire de Chine</li></country></list><tree><country name="République populaire de Chine"><noRegion><name sortKey="Ma, Yan" sort="Ma, Yan" uniqKey="Ma Y" first="Yan" last="Ma">Yan Ma</name></noRegion><name sortKey="Liu, Dingsheng" sort="Liu, Dingsheng" uniqKey="Liu D" first="Dingsheng" last="Liu">Dingsheng Liu</name><name sortKey="Liu, Peng" sort="Liu, Peng" uniqKey="Liu P" first="Peng" last="Liu">Peng Liu</name><name sortKey="Ma, Yan" sort="Ma, Yan" uniqKey="Ma Y" first="Yan" last="Ma">Yan Ma</name><name sortKey="Ma, Yan" sort="Ma, Yan" uniqKey="Ma Y" first="Yan" last="Ma">Yan Ma</name><name sortKey="Wang, Lizhe" sort="Wang, Lizhe" uniqKey="Wang L" first="Lizhe" last="Wang">Lizhe Wang</name><name sortKey="Yuan, Tao" sort="Yuan, Tao" uniqKey="Yuan T" first="Tao" last="Yuan">Tao Yuan</name><name sortKey="Yuan, Tao" sort="Yuan, Tao" uniqKey="Yuan T" first="Tao" last="Yuan">Tao Yuan</name><name sortKey="Zhang, Wanfeng" sort="Zhang, Wanfeng" uniqKey="Zhang W" first="Wanfeng" last="Zhang">Wanfeng Zhang</name><name sortKey="Zhang, Wanfeng" sort="Zhang, Wanfeng" uniqKey="Zhang W" first="Wanfeng" last="Zhang">Wanfeng Zhang</name></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/CyberinfraV1/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000421 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 000421 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= CyberinfraV1
|flux= Main
|étape= Exploration
|type= RBID
|clé= ISTEX:6B02BA7B32D997BB4BE8B5F238D7607ECDB8CDAB
|texte= Distributed data structure templates for data‐intensive remote sensing applications
}}
| This area was generated with Dilib version V0.6.25. |  |